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ARTICLE 

Remote Live Invigilation: A Pilot Study 

Mariana Lilley*, Jonathan Meere f and Trevor Barker* 


There has been a growth in online distance learning programmes in Higher Education. This has led to an 
increased interest in different approaches to the assessment of online distance learners, including how 
to enhance student authentication and reduce the potential for cheating in online tests. One potential 
solution for this is the use of remote live invigilation. This work reports on a small scale pilot study 
where a group of 17 online distance learning Computer Science students from 7 different countries 
(Egypt, Kenya, Saudi Arabia, Slovakia, Trinidad & Tobago, United Kingdom, Zambia) took part in an online 
test using remote live invigilation. Some examinees expressed concerns about data security and privacy. 
Furthermore, some examinees expressed concerns about the extent to which the remote live invigilation 
process would be intrusive, and impact negatively on their online assessment experience. Overall, findings 
from this study suggest that the remote live invigilation did not affect the assessment experience of the 
examinees in any way, with some examinees reporting that knowing that a live proctor was present gave 
them "peace of mind" in case technical problems occurred during the online test. Additionally, examinees 
suggested that remote live invigilation should be used more widely in online distance learning programmes 
as a means to enhance credibility. 
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Introduction 

Upholding academic integrity has always been a priority 
for academic institutions (Spaulding, 2009). However, 
knowing that a student registered on a particular pro¬ 
gramme is the same student sitting a test, and ensuring 
they abide by the institution’s code for academic integrity 
is often viewed as more challenging for online distance 
learning programmes than for their traditional campus- 
based counterparts (Lanier, 2006; Grijalva, 2006; Olt, 
2002; Spaulding, 2009; Ullah et al., 2012). The introduc¬ 
tion of certification for Massive Open Online Courses (Parr, 
2013) has led to an increased interest in student authenti¬ 
cation in online distance learning contexts as exemplified 
by Coursera’s Signature Track (Bartholet, 2013). 

More recently there has been a trend towards the adop¬ 
tion of technologies that support remote invigilation 
(Bartholet, 2013; Eisenberg, 2013). “Remote web invigi¬ 
lation'’ typically refers to invigilation where a webcam is 
used to record examinees during a test; invigilators and/or 
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others can subsequently review the recorded session at a 
convenient time. “Remote live invigilation” typically refers 
to invigilation where a webcam is used to enable human 
proctors to watch and monitor examinees remotely as 
they complete exams. 

This paper summarises the joint work between the 
School of Computer Science and UH Online at the 
University of Hertfordshire (UH) in evaluating the use of 
remote live proctors for the purposes of improved student 
authentication and invigilation in online tests. The follow¬ 
ing were also motivating factors for both parties: 

1. The increased use of online tests on online distance 
learning programmes, and the need to support stu¬ 
dents in adhering more closely to technical instruc¬ 
tions (e.g. logging into the assessment system) as 
well as providing more detailed information to UH 
staff when problems occur. 

2. Unlike in previous years, specific online distance 
learning student groups are now more likely to meet 
the hardware and network requirements of remote 
live invigilation. 

3. The growth of remote invigilation vendors has led 
to more choice in the types of remote invigilation 
available at more competitive price models. 

The key features of the Pilot Study are introduced next. 
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The Pilot Study 

The pilot study was organised into two phases as shown in 

Table 1. 

Phase 1 included two objective tests, one formative and 
one summative. The tests consisted of both multiple- 
choice and multiple-response questions. Both assess¬ 
ments were password protected meaning only the 
proctor from the service provider was able to launch the 
assessment. In addition, the assessment was delivered via 
a secure browser window meaning no other application 
could be launched during the test. 

Phase 2 consisted of one summative test. A summary of 
the tests included in the pilot can be found in Table 2. 

In this work, an external service provider was used 
for the remote live invigilation. For completeness, a 
brief summary of the authentication and invigilation pro¬ 
cess is presented below. 

How the remote live invigilation worked 

There are a number of providers that offer remote live 
invigilation, including ProctorU (2015a) and Software 
Secure (2015). There are a number of administrative steps 
that precede the remote live invigilation. These include 
setting up the test with the service provider, which entails 
providing the necessary details of the examination such as 
dates, times, access information and crucially the exami¬ 
nation rules. In addition, the contact details of the person 
overseeing the exam (usually the module leader) are also 
given to ensure proctors can call upon the institution for 
assistance if required. 

Once a request has been accepted, students are invited 
to book a preferred time slot within the window of oppor¬ 
tunity specified by the module leader. The service is avail¬ 
able 24 hours a day 7 days a week so there is potential 


Phase Module Participants' Country 

of residence 


for exams to take place across the complete range of 
time zones. After a booking has been made, students are 
sent a confirmation email and the module leader is also 
notified. 

On the day of the exam, students are required to log in 
at the time of their chosen slot. An email reminder is sent 
24 hours prior to the exam together with full instructions 
on how to do this or request help if need be. If a student 
forgets to log in then the service provider will contact 
them via telephone. 

Once logged in, students are prompted to download 
and run the software that will connect their webcam and 
desktop to a live proctor. After a connection has been 
established, the following authentication and environ¬ 
ment checks may begin: 

1. The first of these is the identity check, in which 
students are asked to present a government or aca¬ 
demic photo identification card. The proctor checks 
that the photo and name on the card match with 
the person who has scheduled the appointment and 
is present onscreen via webcam. Nothing is recorded 
or captured at this stage. Instead it is simply logged 
as having been checked. 

2. Next, the proctor takes a digital photo of the student 
and stores this on the service provider system for 
future reference. If there is already a photo on file, 
the proctor will compare it to the person onscreen. 

3. US citizens are required to answer a series of 
multiple-choice challenge questions based on public 
records; a typical example would be selecting a 
previous postcode from a choice of four. Non-US 
citizens are typically required to present a second 
form of photo identification. 

4. Following authentication, students are asked to pan 
over their work area using their web cam and hold a 
reflective surface to the camera to ensure there are 
no disallowed materials or persons present. 


1 


2 


Level 6, BSc(Flons) 
Computer Science 


Level 4, BSc(Hons) 
Computer Science 


Kenya 

Saudi Arabia 
Trinidad & Tobago 
United Kingdom 
Zambia 

Egypt 

Kenya 

Slovakia 

United Kingdom 


Table 1: Pilot Study Overview. 


Phase 

Test 

Number of 
participants 

Type 

Duration 

No. of 
questions 

1 

la 

4 

Formative 

30 minutes 

20 

1 

lb 

8 

Summative 

30 minutes 

20 

2 

2 

9 

Summative 

50 minutes 

40 


Table 2: Phases 1 and 2 detailed. 


If all the checks are completed to the satisfaction of the 
proctor, the examination rules are read out by the proctor 
and the student is invited to log into the assessment appli¬ 
cation. Depending on how the assessment is configured, 
the proctor may be required to enter a password to launch 
the assessment on behalf of the student. 

Finally, once the assessment is running, the proctor 
will monitor the student’s desktop, their environment 
(for example, sound) and also their conduct. Any unusual 
behaviour is logged and a response is made in accord¬ 
ance with the specifications provided by the module 
leader. Typically a student is first warned so that they may 
rectify their behaviour, if the behaviour continues then 
the proctor can use their judgment to gather enough 
evidence (photos and screenshots) to report back to the 
institution. 

It should be noted that, prior to the pilot taking place, 
some UK-based participants expressed specific concerns 
about data protection and privacy, for example: 
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• being viewed by “a stranger” and sharing their living 
environment; 

• showing personal identification to “a stranger” and 
the possibility of identity theft; 

• giving “a stranger” remote access to a personal com¬ 
puter and the potential for data to be removed or 
malicious software to be installed; 

• the need for guarantees that the proctoring service 
would adhere to data protection and privacy laws. 

Participation in the pilot study was optional. Participants 
used their University student ID card as the main means for 
test authentication. Furthermore, participants were pro¬ 
vided information relating to the service provider’s safety 
and security certifications as well as their compliance with 
a number of safety and security frameworks (ProctorU 
2015b) before deciding whether or not they should take 
part in the pilot. 

Following the completion of their proctored assessment, 
participants were invited to share their experiences with 
the research team. Nine out of 21 participants replied, and 
the key findings from this evaluation can be found below. 

Participants' Attitude Towards Remote Live 
Invigilation 

In order to gain a deeper understanding of participants’ 
attitude towards remote live invigilation, participants 
were invited to answer three open questions via email. 

1. Does having a live proctor make you feel more sup¬ 
ported should something go wrong during a test 
(e.g. technical problem, sickness etc.)? 

2. You have taken a test with a live proctor. Did you feel 
that having a proctor hinders or enhances online 
assessment in any way? 

3. Do you think we should use remote live invigilation 
in other modules? 

Table 3 summarises participants’ responses to question 1. 
It can be seen from Table 3 that 7 out of 9 participants 
reported that the availability of a live proctor made them 
feel more supported as exemplified by the following quote: 

"Yes, it is preferable to have someone on hand in the 
event of a problem during the exam". 

As can be seen from Table 4, no participant reported that 
the remote live invigilation hindered their online assess¬ 
ment experience. Interestingly, one student said they had 

Question Yes Neutral No 

1. Does having a live proctor make 7 2 0 

you feel more supported should 

something go wrong during a test 

(e.g. technical problem, sickness 

etc.)? 

Table 3: Feeling more supported during assessment 

process (N = 9). 


initial concerns about being watched but, in practice, this 
was not an issue as the invigilation process was largely 
unobtrusive: 

“I don't feel it enhanced my exam, but it did not hin¬ 
der it either. The speed of the exam would not have 
allowed a student to search notes even if they wanted 
to. At the start / was concerned with the thought of 
having someone watching me, not because I could 
not cheat, but because I felt it may have added extra 
pressure. Once I had started I was glad the proctor 
went invisible, so to speak, And I was able to take the 
exam uninterrupted." 

Furthermore, some participants reported that the pres¬ 
ence of a live proctor enhanced their assessment by allow¬ 
ing them to concentrate more and worry less, as captured 
by the following quote: 

"[Remote live invigilation] enhances online assess¬ 
ment because student need not worry about explain¬ 
ing any errors before, during and after the exam. A 
live proctor takes care of that and hence student’s 
level of concentration is higher as a result.” 

Table 5 shows that 8 out of 9 participants were support¬ 
ive of extending the use of remote live invigilation to 
other modules. These participants listed factors such as 
increased credibility of the course as well as the feeling of 
being “looked after” and valued by the institution as the 
main reasons for their support. 

The participant who did not support the use of remote 
live invigilation in other modules commented that the 
authentication process took too long resulting in some 
assessment anxiety: 

“ Usually during this period before an exam I take the 
time to relax and mentally prepare for the exam. In 
this case I felt having to go through this lengthy pro¬ 
cess, made me feel nervous and anxious." 

Question Enhances Neutral Hinders 

2. You have taken a test 2 7 0 

with a live proctor. Did you 

feel that having a proctor 

hinders or enhances online 

assessment in any way? 

Table 4: Perceived impact of remote live invigilation on 
online assessment experience (N = 9). 


Question Yes Neutral No 

3. Do you think we should use remote 8 0 1 

live invigilation in other modules? 

Table 5: Participants’ perceptions as to whether the use 
of remote live invigilation should be extended to other 
modules (N = 9). 
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Summary and Future Work 

The work reported here is concerned with an initial evalu¬ 
ation of remote live invigilation in an online distance 
learning programme. As part of this work, participants 
took place in an online test in their home environment. 
Student authentication and invigilation were carried out 
by a remote live proctor via web conferencing and screen 
sharing technologies. 

Despite the limitations of a small scale study, the results 
reported here indicate that remote live invigilation pre¬ 
sents a potential solution to the issue of student authenti¬ 
cation and cheating in online examinations. Furthermore, 
in spite of some initial concerns about data protection 
and the impact that feeling “watched" might have on their 
online assessment experience, participants’ feedback on 
the use of the remote live invigilation was positive overall 
with some even suggesting that the presence of a proctor 
might reduce stress if things go wrong. 

Often in Higher Education high-stakes assessment is in 
the form of project work and coursework which are sub¬ 
mitted electronically or in hard-copy. Student authentica¬ 
tion in these circumstances often relies on the experience 
of tutors who certify that the work was that of the candi¬ 
date. Any form of assessment is at risk of cheating by, for 
example, impersonation or collusion. It should be noted 
that the security of online examinations does not relate so 
much to keeping out intruders, as in a secure banking sys¬ 
tem, but rather to authenticating the candidate to ensure 
that the test is completed by the candidate and not some 
other person on their behalf. 

The challenges in authenticating candidates and ensur¬ 
ing that they adhere to assessment regulations in online 
assessment have been described by Rowe (2004) and 
Rogers (2006) amongst others. Whilst it may not be possi¬ 
ble to ensure that an online examination system is totally 
secure, it is the view of the authors that remote live invig¬ 
ilation goes some way to providing assurances that the 
person taking the exam is indeed the candidate, and to a 
greater extent that they are working alone and unaided. 

As part of our future work, we plan to carry out a fur¬ 
ther study involving a large cohort in order to gain a deeper 
understanding of students’ attitudes towards remote live 
invigilation. As mentioned earlier this was a small scale 
pilot, and the participants who shared their experiences 
were volunteers and therefore subject to self-selection bias. 

Another area for future work would be to investigate the 
design and use of challenge questions based on specific 
learning contexts (for example, the candidate’s contribu¬ 
tion to a forum discussion) and information known only 
to the candidate to further discourage impersonation. 

Finally, it is hoped that the use of remote live invigila¬ 
tion would allow for greater flexibility in the assessment 


formats used in online distance learning programmes. 

For example, it would make it feasible for online distance 

learning students to take part in timed practical program¬ 
ming tests under supervised conditions. 
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